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ABSTRACT 

This investigation sought to ascertain whether 
significant differences in pupils* achievement when solving 
open- sentence types existed in relation to the following factors: 
school grade, sentence form as determined by the symmetric property 
of the equality relation, the operation specified in a sentence, the 
position of the placeholder in a sentence, and the existence of an 
open— sentence solution within the set of whole numbers. Tables 
present the results of performance of 3,268 pupils in grades 1, 2, 
and 3. (Author/DB) 
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Summary 



Obiectives. For open sentences derived from a well defined domain ofbasxc 
n umber fa ^te of the form aob= c , this investigation ^sought to -cirt^n 
whether significant differences in pupils achievement when solving such sen 
tences existed in relation to the following factors* (A) 9?”*#, J®' 

sentence form as determined by the symmetric property of the ^ 

tion, (C) the operation specified in a sentence, (D) the position of the Place- 
holder in a sentence, and (E) the existence of an open-sentence 
in the set of whole numbers. Table 1 identifies for each of these factors the 
particular levels considered in the present investigation# 

Methods. Based upon factors B, C, and D (Table 1) there may be generated ^ 
the 12 generic open- sentence forms identified in Table 2. Depen ng P°^ 
whole numbers selected as constants, which takes into account factor E , these 
12 generic forms may spawn the 20 particular open-sentence types also xdentx 

£i A ^ 3 2 - itera^ Inventory, Part 1* was developed which included two 
each of the 12 types W1-W12 (Table 2) and one exemplar for each ritteBtype 
X3-X10. Each exemplar was derived by modifying appropriately , n 

or subtraction fact selected from the set jaf such facts having su ^ between 1 
and 18. Inventory Part 1 then was partitioned in a structured way 
8-item Tests which were balanced with respect to levels of factors B, C, D, e 
(T able 1). The placeholder in each ogen sentence was shown as a square re- 
gion (H) rather than as a square (fj) so that each Test could be cast in an 
appealing context of "numbers hiding under boxes." . 

Four additional open sentences— each using numbers less th^n J Tests, 
velpped as a set of sample items cordon to the o res- 

in essence the following instructions were given, wxth simplicity of expres 
sion taking precedence over mathematical preciseness or pedantry: 

"What whole number is hiding under the box? 

~ "write the number on the line. 

"If no whole number is hiding under the box, 
mark a big X on the line." 

Data source. Data were based upon performance of pupils £ ™*^ eS at 
e ach of tte S grade levels-l,2,3-in each of 23 of 37 elementary schools 
using the same city-adopted basal mathematics textbook series. . _ . 

3,268 pupils took one of the four group- administered Tests ic 
tributed randomly among children within each intact mathematics class. 

Design for data analysis. The levels of factor A (grade) may be viewed as 
defining three treatments designated as XI, X2, X3 xn Figure • ' ' 

D,E and their respective levels (Table 1) are inherent xn observation O of 

Fi STfactorial design used to analyze the data is r^res^t^by Figure 2. 
Since it is impossible to completely cross the levels of^ctorsD and E, it 
was not feasible to generate a single ANOVA that embraced all four matnmatxcal 
factors — B , C , D , E — in terms of the Figure 2 model. Consequently, separate 
ANOVAs were generated for particular factor/lavel com binations that could 

* This was in reality a two-part Inventory in which Part 2 also consisted of 
32 items, but of a somewhat different nature: pairs of open s f 1 ^ enc ®® *° 

be judged equivalent or nonequivalent. The present Summary relates only to 

Part 1 of the full Inventory. 
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completely crossed in keeping with the Figure 2 paradigm. The delimiting 
conditions for each of these ANOVAs is made explicit in connection with Table 4 
(to be considered in the next section of this Summary). .... * 

The Figu r e 2 model makes it possible to test the statistical significance o 
the main effect and sundry interactions associated with factor S (school) in the 
case of any AHOVA having more than one replicate per cell. Although 
tion regarding this factor may be of interest to the particular school district 
involved, factor S was of no interest oer sc in relation to the purpose (s) of 
the investigation. Hence no data pertaining explicitly to factor S have been 

included in this report. . . , 

Error terms for the AN OVA model (Figure 2) are specified in Figure 3. 



Results. Table 3 gives an indication of mean correct responses fors 

(1) factor A (grade)— across levels and by levels; and 

(2) levels of mathematical factors B,C,D,E— across and by levels of factor A. 
Table 4 characterizes five AKOVAs that were generated in accord with the Pigur* 

2 model. For each of these ANOVAs the main effects and first-order interaction 
effects are identified, along with a probability value (p) to if^cate the * 

at which Ho for each effect could be rejected on the basis of cci^uted F. ( t 
has not been feasible in this Summary to include the complete AljOVA table for 
each of the five ANOVAs generated.) Note that in Table 4 .10 jS p is used ® 

a broad catch-all category to embrace any instance for which the risk o g 

a Type I error would exceed an o- level of .10. 

More details of the results, their interpretation, and a consideration of im- 
plications of the investigation will be incorporated in the oral presentation. 
Attention will be given to the fact that although there are highly significant 
main effects in connection with the first three ANOVAs (Table 4) , there also are 
some highly significant interactions. Furthermore, the relative occurrence 
of highly significant effects (main and interaction) is not as marked for the 
fourth and fifth ANOVAs as for the first three. 

Findings will be interpreted in relation to pupils* opportunity to learn. 
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TABLE 1 



Factors and Levels for ANOVAs: SOMBER PUZZLES Inventory, Part 1 



Factor 


Level 


„ i 

A. Grade j 

i 


1. First grade 

2. Second grade 

3. Third grade 


B. Symraetric property of the equality relation 


1. Sentence of the 
forn a o b s c 

2. Sentence of the 
fora c = a o b 


C. Operation specified in the open sentence 


1. Addition (+ for o) 

2. Subtraction (- for o) 


D. Position of the placeholder in the sentence 


1. Q in the a position 

2. 0 in the b position 

3. 0 in the c position 


E. Existence of a solution within set X! 


1. A solution exists in W 

2. No solution exists in W 


S. School 


1—23. Schools 1,2,3,. • . ,23 

i 



Note. — Set W = {0, 1, 2, 3, 4, 5, 6, 7, 



.} 
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TABLE 2 



Generic Poms 



and Particular Types of staple Open Addition and Subtraction Sentences 




Notes.— 

# using vhole numbers [B - {0, 1. 2, 3, 4, 5, 6, 7, •••H as constants, it is 
impossible to have an open sentence of this particular type. 

Generic forms 1,2,7 arrf 8 may be referred to as canonical open-sentence forms. 




A 
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TABLE 2.1 



NUMBER PUZZLES Inventory, Part 1 

Distribution of Iteas by Mathematical Factors and Levels and by Open-sentence Types 




Hie above distribution is embedded within 
each of the 3 levels of factor A and with- 
in each of the 23 levels of factor S. 
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The 8 items in Part 1 of each of the 4 Tests were distributed so that • ••• 

1. The same generic open- sentence form (Table 2) was represented by no 
more than one item. 

2. Factors B and C (Table 1) were completely crossed and balanced, with 
2 items for each of the 4 factor/level combinations. 

3. There were 6 W-type sentences (factor E, level 1) s 

3 items for each of the 2 levels of factor B; 

3 items for each of the 2 levels of factor C; 

2 items for each of the 3 levels of factor D. 

[Across the 4 Tests factors B, C, and D were completely crossed 

and balanced, with 2 items for each of the 12 factor/level combi- 
nations.] 

4. There were 2 X-type sentences (factor E, level 2) : 

1 item for each of the 2 levels of factor B? 

1 item for each of the 2 levels of factor C. 

[Across the 4 tests each of the 8 possible B,C,D factor/level com- 
binations was represented once.] 



Separate consideration was given to a balanced distribution (within and 
across Tests) of items with respect to the relative magnitude of sums, 
first addends, and second addends. 
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XI 


(Gl) 0 


X2 


(GlV_/ 02 ) 0 


X3 


(G1V^G2v^G3) 0 



Fig. 1. Research paradigm pertaining to factor A 



Notes. — 

^2, Y3< refer to the 1967-68, 1968-69, 1969-70 school years respectively. 



Gl, G2, G3 refer to the Instructional programs based upon the city-adopted 
mathematics textbook series for grades 1, 2, 3 respectively. 

I 

The same observation, 0 (NUMBER PUZZLES . Inventory) was made for treatments 
XI, X2, X3 in the spring of the 1969-70 school year (Y3). 
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TABLE 3 



Mean Correct Responses Across Schools: NUMBER PUZZLES Inventory, Part 1 



Factor and level 


Number 
of items 


Across 

grades 


Grade 1 


Grade 2 


Grade 3 


A. Grade 


32 


18.12 


12.79 


19.07 


22.51 


B. Symmetric property of * 












1. a o lo ** £ 


16 


10.01 


7.52 


10.70 


11.80 


2. = a, o b 


16 


8.11 


5.27 


8.36 


10.71 


C. Operation specified 












1. Addition (+) 


16 


10.48 


7.32 


11.04 


13.06 


2. Subtraction (-) 


16 


7.65 


5.47 


8.02 


9.44 


D. Placeholder position 












1. Q in a position 


10 


4.42 


2.67 


4.73 


5.86 


2. Q in b position 


12 


7.30 


5.21 


7.65 


9.04 


3. [Zl in £ Position 


10 


6.40 


4.90 


6.69 


7.61 


E. Solution in W 












1. A solution exists 


24 


14.37 


9.47 


15.28 


18.37 


2. No solution exists 


8 

1 


3.75 


3.32 

1 


3.79 


4.13 
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TABLE 3.1 



Mean Correct Responses Across Schools : NUMBER PUZZLES Inventory , Part 1 



Generic form of 


Sentences having a solution 
within set W [2 items/type J 


Sentences having no solution 
within set W [1 item/type] 


open sentence 


Grade 1 


Grade 2 


Grade 3 


Grade 1 


Grade 2 


Grade 3 


1 


a + b « f~) 


1.28 


1.74 


1.86 


— 


— 


— 


2 


□ = a + b 


.99 


1.42 


1.75 


— 


— 


— 


3 


a + D B c 


1.06 


1.57 


1.84 


.46 


.53 


.51 


4 


c = a + Q 


.80 


1.34 


1.71 


.38 


.53 


.63 


5 


□ + b - c 


.90 


1.51 


1.76 


.43 


.60 


.59 


6 


c - □ + b 


.63 


1.33 


1.69 


.37 


CO 

KP 

• 


.63 


7 


a - b = □ 


1.13 


1.59 


1.75 


.49 


in 

• 


.46 


8 


□ = a - b 


.70 


1.21 

* 


1.49 


.32 


.27 


.29 


9 


a- □ -c 


I 1,00 


1.59 


1.79 


.54 


in 

• 


.49 


10 


c ° a - □ 


1 .64 


1.15 


1.53 


.33 


.36 


.44 


11 


0 
u 

XX 

1 

□ 


.23 


.55 


.65 




— 


--- 


12 


0- □ -b 


.10 


.26 


.55 


1 “ 
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Fig. 2. ANOVA model: 3 x p x q x ...x23 

(Repeated measures design) 
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Notes. — 

This is a mixed model involving n factors, completely crossed, where: 

A (grade) is a fixed factor (3 levels); 

J (some mathematical characteristic) is a fixed factor (p levels); 

K (some mathematical characteristic) is a fixed factor (q levels), 

(any other mathematical characteristics as fixed factors); and 

S (school) is considered to be a random factor (23 levels) . 



The number of replicates is the same for each cell of the matrix but may differ 
from one ANOVA to another. In any case, each replicate is a school gffi* 
mean correct responses for all pupils from a particular school and grade (pooled 
across classes) on a particular set of Inventory items. 



ANOVAs computed on the basis of the above design were run at the Stanford 
University Computing Center using the BMD08V Analysis of Variance program 
revised July 17, 1969 by the UCLA Health Sciences Computing Facility. 
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Error term 


Source of 
variation 


df 






r = 1 


r > 1 






A 


a ■ 1 » 3 ■ 1 = 2 


AS 


AS 


J 


j " 1 


JS 


JS 


K 


k - 1 


KS 


KS 


S 


8-1 = 23-1 = 22 


— 


R(AJKS) 


AJ 


(a - 1) (j - 1) 


AJS 


AJS 


AK 


(a - 1) (k - 1) 


AKS 


AKS 


AS 


(a - 1) (s - 1) 


— 


R(AJKS) 


JK 


( j - 1) (k - 1) 


JKS 


JKS 


JS 


( j - 1) (s - 1) 


— 


R(AJKS) 


ICS 


(k - 1) (s - 1) 


— 


R(AJKS) 


AJK 


(a - 1) (j - 1) (k - 1) 


AJKS 


AJKS 


AJS 


(a - 1) (j - 1) (s - 1) 


— 


R(AJKS) 


AKS 


(a - 1) (k - 1) (s - 1) 


— 


R(AJKS) 


JKS 


(j - l)(k - 1) (s - 1) 


— 


R(AJKS) 


AJKS 


(a - 1) (j - 1) <k - 1) (s - 1) 


— 


R(AJKS) 


R(AJKS) 


(r - l)ajks 


— 




Total 


rajks - 1 «=> N - 1 







*Fig. 3. 



Error terms for ANOVA model. Fig. 2. 



Notes. ~ 



The preceding patterns may be extended, of course, to include additional 
fixed factors. 

We may view R . . 'replication* within the smallest cell of a design 
[as] a nested factor that is always random and is nested within all the 
other factors of the design." (Glass & Stanley, 1970; p. 474) 

The "within cells" source of variation, R(AJKS), is nonexistent when 
r - 1. 



Oopsl This is a table rather than a figure. Sorry about that. 
The mistake will be corrected in any future document. 
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TABLE 4 



Significance Levels for Main Effects and for Two-factor Interactions. 
NUMBER PUZZLES Inventory, Part 1 AKOVAs 




* 



TABLE 4 (Continued) 



3. ANOVA based on 8 sentences having no solution in w 
(Factor E, Level 2) 




Main effects A 


.01 >. p > .001 


B 


p .001 


C 


p <_ .001 


Interactions AB 


p .001 


AC 


p .001 


BC 


p .001 


4. ANOVA based on 4 addition sentences (Factor C, Level 1] 
having no solution in W (Factor E, Level 2] 




Main effects A 


p .001 


B 


.10 < p 


D 


.10 < p 


Interactions AB 


.10 >. P > *05 


AD 


.10 < p 


BD 


.10 < p 


5. ANOVA based on 4 subtraction sentences (Factor C, 
Level 2) having no solution in W (Factor E, Level 2] 




Main effects A 


.10 < P 


B 


p < .001 


D 


.01 > P > .001 


Interactions AB 


.10 < p 


AD 


.10 < p 


BD 


.10 < P 
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5.1 



NUMBER PUZZLES Inventory, Part 1. 

Comparison of P’s with Factor A Included and Excluded i 
ANOVA based on all 32 sentences 
[ANOVAs 13 and 13. ly Table 17 (4.1) J 




If no coding is associated with a reported F, p > .10 



r 

i : 

!’■ 
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5.2 



NUMBER PUZZLES Inventory, Part 1. 

Comparison of F f s with Factor A Included and Excluded; 

ANOVA based on 24 sentences having solutions in w (Factor E # Level 1) 
[ANOVAs 8 and 8.1} Table 19 (4.2)] 



Source of 
variation 




F 


df for F 


Factor A 


Factor A excluded 


included 


Grade 1 


Grade 2 


Grade 3 


A 


2,44 


246.86*** 








B 


1,22 


399.82*** 


159.80*** 


169.88*** 


84.33*** 


c 


1,22 


669.83*** 


110.02*** 


213.97*** 


906.48*** 


D 


2,22 


698.72*** 


117.61*** 


353.11*** 


312.79*** 


BC 


1,22 


9.75** 


.41 


7.73* 


9.30** 


BO 


2,44 


10.17*** 


6.16** 


4.22* 


4.03* 


CD 


2,44 


384.57*** 


28.66*** 


180.44*** 


439.46*** 


BCD 


2,44 


2.26 


2.13 


.68 


.84 


AB 


2,44 


15.17*** 








AC 


2,44 


11.50*** 








AD 


2,88 


4.38** 








ABC 


2,44 


1.13 








ABD 


4,88 


.14 








ACD 


4,88 


19.00*** 








ABCD 


4,88 


1.02 









* .10 > p > .05 * .05 > p > .01 ** .01 > p > .001 *** P < .001 

It no coding is associated with a reported F , p > .10 
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NUMBER PUZZLES Inventory, Part 1. 5.3 

Comparison of P's with Factor A Included and Excluded: 

ANOVA based on 8 sentences having no solution in VJ (Factor E, Level 2) 
[ANOVAs 6 and 6.1; Table 21 (4.3)] 



Source of 
variation 


df for F 


F 


Factor A 
included 


Factor A excluded 


Grade 1 


Grade 2 


Grade 3 


A 


2,44 


6.86** 








B 


1,22 


79.79*** 


48.99*** 


49.78*** 


9.22** 


C 


1,22 


53.19*** 


.020 


41.89*** 


75.51*** 


BC 


1,22 


20.40*** 


3.10# 


7.09* 


8.99** 


AB 


2,44 


11.57*** 








AC 


2,44 


27.09*** 








ABC 


2,44 


.024 









# .10 >. p :♦ .05 * .05 >. p > .01 ** .01 > p > .001 *** p ± .001 

If no coding is associated with a reported F, p > .10 





! 
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